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PREFACE 

This  report  summarizes  research  performed  for  the  Naval  Training  Equip¬ 
ment  Center,  Orlando,  Florida,  under  contract  N61339-78-C-0136.  Two  pri¬ 
mary  research  tasks  v/ere  completed  under  the  general  rubric  of  a  "Training 
Improvements  Pi’ogram"  for  the  Navy's  Tactical  Aircrew  Combat  Training 
System  (TACTS).  A  summary  of  work  completed  on  two  major  task  areas, 

(1)  Development  of  a  Computer  Based  TACTS  Debrief  System,  and  (2)  A 
Problem  Definition  Study  of  Navy  Missile  Envelope  Recognition  Training,  is 
presented  herein.  It  should  be  noted  that  two  more  technically  detailed 
reports,  covering  the  same  research  and  development  tasks,  have  already 
been  published.  Individuals  requiring  greater  detail  on  research  background, 
empirical  findings,  and  technical  discussion,  should  consult  these  earlier 
studies  which  are  referenced  in  this  summary  report. 
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SECTION  I 
INTRODUCTION 


BACKGROUND 

This  report  reviews  research  and  development  efforts  related  to  a  Train¬ 
ing  Improvements  Program  for  the  Navy's  Tactical  Aircrew  Combat  Training 

System  (TACTS).  The  research  reported  traces  its  origins  to  previous  studies 

12  3 

of  air  combat  performance  measurement  for  TACTS.  ’  ’ 

The  TACTS,  formerly  called  the  Air  Combat  Maneuvering  Range  (ACMR), 
is  an  instrumented  system  used  primarily  to  train  Navy  aircrews  in  air-to-air 
combat.  Briefly,  TACTS  provides  a  capability  for  real-time  tracking  of  air¬ 
craft  engaged  in  air  combat.  The  system  has  a  replay  capability  which  pro¬ 
vides  a  rich  source  of  quantitative  information ,  including  inter-aircraft  position 
data  and  computer  generated  weapon  launch  outcomes. 

The  availability  of  TACTS  data  has  provided  a  rare  opportunity  for 
researchers  to  obtain  operational  measures  of  aircrew  pe  formance.  These 
measures  have  potential  application  across  a  variety  of  ongoing  aviator  selec¬ 
tion  and  training  research  efforts. 


^Ciavarelli,  A.P.,  Brictson,  C.A.,  and  Young,  P.A.  Development  and  Appli¬ 
cation  of  Performance  Criteria  and  Aircrew  Assessment  Methods  for  the  Air 
Combat  Maneuvering  Range  (ACMR)  (U).  Pensacola,  Florida:  Naval  Aero- 
space  Medical  Research  Laboratory,  Special  Report  79-5,  September  1979. 
(CONFIDENTIAL) 

2 

Brictson,  C.A.,  Ciavarelli,  A.P.,  Pettigrew,  K.W.  and  Young,  P.A.  Perfor- 
m ance  Assessment  Methods  and  Criteria  for  the  Air  Combat  Maneuvering 
Range  (ACMR):  Missile  Envelope  Recognition  (CTT  Pensacola,  Florida:  Naval 
Aerospace  Medical  Research  Laboratory,  Special  Report  No.  78-4,  July  1978. 
(CONFIDENTIAL) 
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Brictson,  C.A.,  Ciavarelli,  A.P.  and  Jones,  T.N.  Development  of  Aircrew 
Performance  Measures  for  the  Air  Combat  Maneuvering  Range  (ACMR)  (U|). 
Pensacola,  Florida:  Naval  Aeroscace  Medical  Research  Laboratory,  Report 
No.  L53001 ,  June  1977.  (CONFIDENTIAL) 
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The  Naval  Aerospace  Medical  Re 'ie arch  Laboratory,  realizing  the  value 

of  having  available  operational  measurement  of  aircrew  performance,  sponsored 

a  program  designed  to  develop  performance  criteria  and  assessment  methods 

for  TACTS.  During  the  course  of  this  measurement  research  program,  two 

4 

significant  fleet  training  deficiencies  w^re  identified: 

1.  Need  for  training  debrief  aid  —  TACTS  training  was 
conducted  in  an  unsystematic  manner,  with  no  stan¬ 
dardized  training  procedures  or  structured  debrief 
aids,  and  no  means  to  provide  performance  based 
training  feedback. 

2.  Need  for  improved  envelope  recognition  training  — 
measures  of  missile  envelope  recognition  performance 
for  Navy  aircrews,  taken  on  TACTS,  showed  consis¬ 
tent  deficiencies  when  compared  to  recommended  fleet 
standards. 

On  the  basis  of  numerous  on-site  observations,  backed  by  empirical  findings, 
it  was  concluded  that  "...the  TACTS  represents  a  significant  technological 
advance  in  instrumentation  and  computing,  but  it  has  not  had  the  benefit 
of  an  organized  training  and  performance  assessment  program." 

TRAINING  IMPROVEMENTS 

The  primary  aim  of  the  original  measurement  research  program  sponsored 
by  the  Naval  Aerospace  Medical  Research  Laboratory  was,  and  continues  to  be, 
the  development  of  reliable  air  combat  performance  criteria  for  use  in  validat¬ 
ing  aviator  selection  variables.  With  the  identification  of  TACTS  training 
deficiencies,  however,  the  more  immediate  benefits  of  applying  these  perfor¬ 
mance  criteria  directly  to  operational  ACM  training  was  recognized. 

A  TACTS  Training  Improvements  Program,  therefore,  was  initiated  and 
sponsored  by  the  Naval  Training  Equipment  Center  in  order  to  focus  some  of 
the  research  results  thus  far  obtained  on  immediate  operational  training  pro¬ 
blems.  The  training  improvements  program  has  concentrated  in  two  areas: 


^See  footnotes  2  and  3,  page  5 
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1)  development  of  a  computer  based  TACTS  debrief  system,  and  2)  a  problem 
definition  study  of  Navy  missile  envelope  recognition  training. 

Research  related  to  these  two  areas  has  already  been  extensively  dis¬ 
cussed  in  earlier  technical  reports.^’®  This  summary  report  presents  high¬ 
lights  of  this  prior  research,  and  in  addition,  includes  some  work  completed 
subsequent  to  these  earlier  studies. 

This  summary  report  has  two  major  sections,  each  covering  one  of  the 
principal  research  task  areas.  Section  II,  The  Performance  Assessment  and 
Appraisal  System  (PAAS),  summarizes  work  completed  to  date  on  a  computer 
based  debrief  system  for  TACTS.  Section  III,  Review  of  Air-to-Air  Missile 
Envelope  Recognition  Training  Problems,  summarizes  the  unclassified  portion 
of  a  more  comprehensive  study.  7  The  reader  interested  in  greater  technical 
detail  than  reported  here  should  consult  these  earlier  reports  (referenced 
below ) . 


SCiavarelli,  A.P.,  Pettigrew,  K.W.,  and  Brictson,  C.A.  Development  of  a 
Computer  Based  Air  Combat  Maneuvering  Range  Debrief  System:  interim 
Report  (Volume  I)~I  La  Jolla,  California:  Dunlap  and  Associates,  Inc., 
January  1986. 


^Ciuvarelli,  A.P.,  Narsete,  E.M.,  and  Brictson,  C.A.  A  Problem  Definition 
Study  of  U.S.  Navy  Air-to-Air  Missile  Envelope  Recognition  Training  (U). 
Interim  Report  (Volume  II).  La  Jolla,  California:  Dunlap  and  Associates, 
Inc.  .  April  1980.  (CONFIDENTIAL) 

7Ibid. 
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SECTION  II 

PERFORMANCE  ASSESSMENT  AND  APPRAISAL  SYSTEM  (PAAS) 

OVERVIEW 


For  the  past  five  years,  scientists  at  Dunlap  and  Associates,  Inc., 
Western  Division,  have  been  intimately  involved  in  the  study  of  fir  combat 
training  and  performance  measurement  on  the  Navy's  TACTS  operated  out  of 
Miramar  Naval  Air  Station,  San  Diego,  California  On-site  observations  made 
during  this  time  and  verified  by  data  collected  from  over  300  engagements 
reveal  that  TACTS  training  missions  and  their  associated  debriefs  vary  con¬ 
sider  t’jly  in  training  emphasis ,  content ,  and  quality .  Additionally ,  cumulative; 
iaK  .  it  routinely  collected,  so  that  performance  review  of  more  than  one 
iSiii'.i  tit  a  time  is  not  readily  available.  Determining  Air  Combat  Maneuver- 
■.i.  .ACM/  training  progress  on  TACTS  is,  therefore,  a  time  consuming  and 
c  pensive  }  recess  in  which  data  from  many  missions  must  be  hand-compiled. 

i l  the  basis  of  our  observations  and  research,  and  in  response  to  requests 
by  \C<!V!  training  personnel,  we  proposed  the  development  of  a  computerized 
syst -n.  PAAS,  to  aid  in  immediate  aircrew  debrief  and  to  facilitate  longer 
term  evaluation  of  training  progress.® 

Thus  far,  we  have  completed  a  single  software  module,  or  subsystem  of 

PAAS.  This  initial  module  represents  a  preliminary,  or  baseline,  TACTS 

debrief  system  which  was  first  introduced  at  the  1980  TACTS  User  Confer- 
9  1  0 

ence.  *  This  baseline  debrief  was  developed  after  several  years  of  research 
using  a  system  approach  to  air  combat  training  and  performance  measurement. 


8See  footnote  5,  paqe  7. 

^Ciavarelli,  A.P.  Development  and  application  of  aircrew  performance  criteria 
and  assessment  methods  for  TACTS.  Proceedings  of  the  Seventh  TACTS/ 
ACMI  Users  Seminar.  San  Diego,  California :  COMNAVAIRSYSCOM, 

May  20-21,  1980. 


^Williams,  A.M.  Performance  assessment  and  combat  effectiveness  (PACE). 
Proceedings  of  the  Seventh  TACTS /ACMI  Users  Seminar.  San  Diego, 
California:  COMNAVAIRSYSCOM,  May  20-21,  ldSd. 
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Using  this  approach,  an  air  combat  engagement  was  considered  to  be  a  system 
composed  of  several  mission  phases ,  and  measures  associated  with  successful 
completion  of  these  phases  were  found  to  be  statistically  related  to  successful 
engagement  outcomes.^  Thus,  the  air  combat  sequence  is  a  highly  structured 
event,  and  overall  ACM  success  depends  on  success  at  several  critical  points. 
With  the  identification  of  these  phases  and  empirical  results  from  ongoing  mea¬ 
surement  studies,  Dunlap  and  Associates  was  ready  to  move  from  the  research 
arena  into  applications,  and  the  computer  based  aircrew  debrief  program  was 
introduced.  The  purpose  of  the  PAAS  is  two-fold.  First,  it  aids  in  structur¬ 
ing  and  therefore  standardizing  the  aircrew  debrief  by  providing  feedback 
for  only  the  critical  ACM  phases  identified  by  our  earlier  research.  And 
second,  training  progress  can  be  observed  because  the  system  stores  data 
from  past  missions,  building  a  cumulative  data  base.  In  addition,  PAAS  allows 
aircrews  to  review  training  results  off-line  to  TACTS  in  a  timely  manner  and 
in  a  summarized  graphic  format . 

The  currently  completed  module,  Performance  Assessment  and  Combat 
Effectiveness  (PACE),  is  a  fully  operational,  stand-alone  system  designed  to 
provide  performance  review  at  the  squadron  level.  The  system  has  been 
developed,  however,  with  an  eye  toward  adding  three  more  modules  to  com¬ 
plete  the  proposed  PAAS  system,  one  module  to  review  individual  pilot  perfor¬ 
mance  (TAD;  Training  Appraisal  and  Diagnostics),  another  to  review  fleet 
normative  data  (NORM:  Numerical  Operational  Readiness  Measurement),  and 
a  third  to  provide  a  complete  set  of  operating  procedures  (SOP:  Standard 
Operating  Procedures).  Figure  1  shows  the  conceptual  design  configuration 
of  PaAS,  with  the  now  completed  PACE  module  highlighted. 


11 
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Figure  1.  Conceptual  Design  of  Performance  Assessment 
and  Appraisal  System  (PAAS) 
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•JACE  has  been  enthusiastically  accepted  by  operational  personnel  at 
Miramar  Naval  Air  Station.  A  letter  of  operational  support,  for  example, 
stated : 

"...  Because  it  is  so  difficult  to  give  the  Commanding  Officer 
readily  available  and  easily  understood  factual  information  on 
which  to  base  his  assessment  of  aircrew  readiness,  I  have 
been  very  favorably  impressed  with  the  PAAS  capability  to 
provide  empirical  data  in  a  meaningful  display.  There  is  a 
great  requirement  for  this  type  of  system,  and  I  strongly 
recommend  continuance  of  the  development  of  the  PAAS 
system. . 

The  letter  has  uaen  forwarded  to  Naval  Air  Systems  Command,  Washington, 
D.C.  and  NAVTRAEQUIPCEN,  Orlando  from  Commander  Fighter  Airborne 
Early  Warning  Wing,  U.S.  Pacific  Fleet  (June  27,  1980,  letter  50/HRK:pw, 
3500,  Ser  1079). 

GENERAL  DESCRIPTION  OF  PACE. 

Application .  PACE  has  been  designed  primarily  for  use  by  the  squadron 
training  officer  to  review  performance  of  a  squadron,  as  a  whole,  for  key 
ACM  tasks.  Missions  to  be  reviewed  are  selected  by  entering  squadron, 
mission  type ,  adversary  aircraft  type ,  and  two  inclusive  dates  in  response 
to  prompts  by  the  computer.  Performance  can  be  reviewed  either  on  a  daily 
basis  by  entering  the  same  date  twice,  or  over  a  more  extended  period  of 
time  by  entering  both  start  and  finish  dates.  Thus,  PACE  could  be  used  to 
review  daily  training  missions  flown  by  the  members  of  a  squadron  to  look 
for  improvements  in  training  on  a  day-to-day  basis,  or  it  could  be  used  to 
compare  performance  among  several  squadrons.  Performance  of  squadrons 
for  entire,  week-long  detachments  car.  also  be  reviewed  and  compared. 
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Computer  Graphics.  Feedback  is  provided  by  PACE  for  several  selected 

training  objectives:  1)  radar  procedures,  2)  look-out  procedures,  3)  tactics 

and  maneuvers,  and  4)  envelope  recognition.  In  our  previous  research,^ 

we  have  defined  these  training  objectives  and  have  demonstrated  their 

importance  to  air  combat  success.  Our  empirical  evidence  has  shown,  for 

example,  that  obtaining  an  early  radar  contact  and  Tally  Ho,  results  in  a 

1 3 

greater  likelihood  of  winning  an  air  combat  engagement.  Figure  2  shows 
the  typical  air  combat  mission  sequence  and  identifies  the  key  events  in  the 
mission,  i.e.,  radar  contact,  Tally  Ho,  etc.,  for  which  performance  informa¬ 
tion  is  required.  We  have,  therefore,  structured  the  PACE  module  to  provide 
graphic  performance  feedback  addressed  to  the  above  four  training  objectives 
and  related  to  most  of  the  key  air  combat  events  depicted  in  Figure  2.  Avail¬ 
able  graphics  for  the  PACE  module  are  itemized  and  discussed  in  the  following 
subsections  of  this  report.  All  data  used  in  graphic  displays  presented  here 
are  hypothetical ,  but  in  actual  application  PACE  uses  data  collected  during 
TACTS  missions. 

Radar  Procedures.  Performance  feedback  on  radar  procedures  is  provided  in 
terms  of  what  percent  of  the  total  number  of  missions  the  fighter  aircrews 
obtained  a  radar  contact  on  the  adversary.  This  percentage  is  further  broken 
down  into  what  percent  of  the  missions  obtained  early  and  late  contact.  (See 
Figure  3) 

Look-Out  Procedures.  Feedback  on  look-out  procedures  is  provided  in  the 
same  graphical  format  as  for  radar  procedures.  The  percent  of  total  missions 
obtaining  a  visual  contact  (Tally  Ho)  and  a  breakdown  in  terms  of  early  and 
late  are  provided.  (See  Figure  4) 


^See  footnotes  1  and  3,  page  5 
13See  footnote  5,  page  7 
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Figure  2.  ACM  mission  sequence 
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f i" ihIiPoh :  pill  from  uiiim,  rs 

MISSION  Ti'F'E:  2V2  TO  14MrtV7S 

HDV.  fi  C:  FILL 


//=?/=? 


cr  cp  Ay  ~r/=f  cr  r~ 


Early  Late  4  *3 

Total  60 


7  3^ 


ERRLY  LRTE  MONE 

'  15  nm  <  1 5  nm 


Figure  3.  Sample  display  format  for  Radar  Contact:  percent  of 
engagements  with  early,  late  and  no  radar  contact 
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mission  type; 
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TO  MI1PV78 


~T'  /=)  /  i__  V"  /W  O  1 


Figure  4.  Sample  display  format  for  Radar  Contact:  potent  of 
engagements  with  early,  late,  and  no  Tally  Ho 
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Tactics.  Tactical  feedback  provided  by  PACE  includes  the  percent  of  missions 
on  which  a  first  shot,  and  independently,  a  first  kill  were  obtained  by  the 
fighter  aircraft.  (See  Figure  5) 

Envelope  Recognition  (Kill/No  Kill).  Missile  fire  envelope  boundaries  are 
defined  in  terms  of  AOT  (angle  off  adversary's  tailpipe)  and  range  (distance 
from  adversary)  at  the  time  of  missile  launch.  The  firing  envelope  defines 
the  boundary  values  for  these  parameters.  To  maximize  changes  of  a  success¬ 
ful  launch,  a  missile  should  be  fired  within  these  boundaries.  PACE  displays 
graphical  feedback  of  AOT  and  range  at  the  time  of  each  missile  shot.  The 
fighter's  position  is  plotted  as  a  star  or  a  circle  (hit  or  miss,  respectively, 
according  to  preselected  rules  for  scoring)  on  a  polar  plot  with  the  adver¬ 
sary's  tailpipe  at  0°,  0  ft.  Missile  shot  fire  points  are  plotted  on  the  so 
called  "hot"  and  "cold"  sides  of  the  hypothetical  missile  envelope.  These 
terms  are  extensively  defined  in  earlier  technical  publications^  and  are 
roughly  equivalent  to  shots  fired  on  the  cockpit  side  (hot)  and  bellyside 
(cold)  of  the  target  aircraft.  Separate  displays  are  presented  for  three 
missile  types,  AIM  9G,  AIM  7F,  and  AIM  9L.  To  score  shots  according  to 
rule-of-thumb  envelopes,  the  computer  overlays  an  outline  of  the  miBsile  fire 
envelope  on  the  polar  plot.*  (See  Figures  6,  7  and  8.)  Missile  fire  positions 
in  Figures  6-8  are  hypothetical  and  used  for  illustrative  purposes  only. 

Engagement  Outcomes.  A  final  display  to  indicate  win/loss  statistics  (i.e.  , 
exchange  rate)  has  been  included  in  the  program.  The  total  exchange  rate 
is  indicated  in  terms  of  total  fighter  hits  to  total  adversary  hits.  Missile 
fire  success  rate  in  terms  of  percent  hits  is  displayed  for  four  missile  types, 
three  fighter  missiles  (AIM  7F,  AIM  9L,  and  AIM  9G)  and  one  type  of  adver¬ 
sary  missile  (ATOLL).  (See  Figure  9) 


l^See  footnotes  2  and  3,  pege  5 

♦Actual  missile  envelopes  ave  classified  and  can  not  be  shown  here  (examples 
are  provided  in  Reference  5) . 
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SCiijhDRON:  ALL  FROM  01Mfi,'78 
MISSION  TYPE:  TO  l«MftVT8 
flBV.  M-C:  ALL 


7“  d'  T~  JT  Ci:  S 


TOTAL  -  60  ENGAGEMENTS 


Figure  5.  Sample  display  format  for  Tactics:  percent  first  fighter  shot 
and  percent  first  fighter  kill  out  of  total  engagements 
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SPUHPRON:  Nll 

FROM 

0IMRV7S 

MISSION  type:  i\'2 

TO 

1 4MHY76 

«DV.  fl.c:  ALL 

MISSlE  T.FE  =  % 


E N VEL OPE  RECOGNI T I  ON 

Range  Cxi Q00  ft.) 


o 


SUCCESS  ROTE  -  ”  -  28  *. 


Figure  6.  Sample  display  format  for  Envelope  Recognition:  Sidewinder 
AIM  9G  --  shows  fighter  aircraft  position  at  missile  fire  in 
terms  of  range  (xlOOO  ft.)  and  angle  off  tail  (AOT  in  degrees) 
from  target.  Circles  and  stars  indicate  hits  and  misses, 
respectively,  according  to  preselected  rules  for  scoring. 
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FROM  OIHRY78 
TO  I4HRV78 

RJV.  R-'C  *  RLl 

MISSLE  TYPE ' =  ?F 


■iOUADRON:  ALL 

RISSION  TYPE!  2V2 


-130 


o  4  3  2 

-90  | - 1 - 1 - H 


\ 


COLD  SIDE 


-40 

13 

SUCCESS  RATE  -  —  -76  *; 

17 


Range  ( x  1  nm. ) 


130 


°  / 

/  1* 

t>4  2  3  4  0 

0  •+* - 1 - 1 - 1 - 1  90 

\  * 

* 

*  *  » 

V 

\ 


HOT  SIDE 


40 


Figure  7.  Sample  display  format  for  Envelope  Recognition:  Sparrow 
AIM  7F  --  shows  fighter  aircraft  position  at  missile  fire  in 
terms  of  range  (n.m.)  and  angle  off  tail  (AOT  in  degrees) 
from  target.  Circles  and  stars  indicate  hits  and  misses, 
respectively,  according  to  preselected  rules  for  scoring. 
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SOUADPON:  ALL  FROM  01MAY79 
MISSION  TYPE:  2V2  TO  MMA1T8 
ADV.  A'C:  ALL 

MIS3LE  TYPE  =  % 


Figure  8.  Sample  display  format  for  Envelope  Recognition:  Sidewinder 
AIM  9L  —  shows  fighter  aircraft  position  at  missile  fire  in 
terms  of  range  (n.m.)  and  angle  off  tail  (AOT  in  degrees) 
from  target.  Circles  and  stars  indicate  hits  and  misses, 
respectively,  according  to  preselected  rules  for  scoring. 
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rate: 


Figure  9.  Sample  display  format  for  Engagement  Outcomes:  shows 
hypothetical  missile  fire  success  rate  (percent)  by  missile 
type  and  overall  exchange  rate  (fighter  to  adversary  (ADV) 
kills) 
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Graphics  Use.  Graphic  displays  can  be  requested  individually,  or  they  can 
be  presented  as  part  of  an  automated  sequence  and  used  to  structure  and 
standardize  the  aircrew  debriefing  procedure.  Hardcopy  printout  of  each 
display  is  also  available  for  permanent  records,  to  make  comparisons  among 
squadrons,  and  to  make  comparisons  over  time  to  determine  training  progress. 

Emrelope  Recognition  (by  engagement).  One  final  graphics  display  is  avail¬ 
able  and  is  used  for  more  detailed  training  data  analysis.  This  display  allows 
the  squadron  training  officer  to  identify  shots  by  engagement.  The  format  of 
this  display  is  the  same  as  the  format  of  Envelope  Recognition  (kill /no  kill). 

It  is  a  polar  plot  with  the  adversary's  tailpipe  at  0°,  0  ft.  Missile  fire 
positions,  however,  are  indicated  by  numbers  rather  than  stars  and  circles. 
These  numbers  refer  to  the  engagement  number  of  each  shot.  Use  of  this 
display  along  with  a  data  listing  feature  incorporated  into  PACE  allows  the 
squadron  training  officer  to  call  up  the  data  associated  with  a  particular  shot. 
(See  Figures  10  and  11  )  This  display  has  an  additional  feature.  It  can  be 
used  to  examine  training  progress  on  a  longitudinal  basis.  Engagements  are 
entered  into  the  data  base  chronologically;  low  numbers  indicate  missile  shots 
taking  place  during  earlier  engagements.  Hence,  comparisons  of  the  plotted 
numbers  from  low  to  high  reveal  whether  or  not  training  improvement  is 
occurring  in  terms  of  shot  accuracy. 

Data  Listing.  This  feature  of  PACE,  which  allows  access  to  all  of  the  engage¬ 
ments  selected  for  debriefing,  displays  all  data  related  to  one  record  in  the 
data  base.  One  record  is  stored  in  the  data  base  for  each  engagement  flown 
by  each  pilot.  The  data  are  listed  on  the  CRT  display  only.  To  preserve 
individual  confidentiality  and  for  security  purposes,  a  hardcopy  of  these  data 
is  not  available.  A  sample  of  a  single  data  record  is  included  as  Figure  12. 
Using  the  information  on  the  record,  such  other  shot  parameters  as  closing 
velocity,  pointing  angle,  and  shooter-indicated-airspeed  are  also  available  for 
debriefing  if  required. 
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SOUABRON!  ALL 
MISSION  TYPE:  £Vi 
ADV.  A'C!  ALL 

HIS5LE  TYPE  *  7F 


PROM  8 1 MAY78 
TO  14MAYP8 


Figure  10.  Sample  display  format  for  Envelope  Recognition  (by  engagement) 
for  selected  missile:  shows  fighter  aircraft  fire  position  in  terms 
of  range  (n.m.)  and  angle  off  tail  (AOT  in  degrees)  from  target. 
Numbers  refer  to  engagement  number  of  each  shot. 
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ENVELOPE  RECOGNITION 


RECORD 

# 

1 

ROT 

117 

RflNGE(ft-) 

18996 

RECORD 

« 

5 

9 

7179 

RECORD 

# 

1 1 

136 

7838 

RECORD 

* 

17 

59 

14279 

RECORD 

# 

20 

55 

12091 

RECORD 

« 

27 

107 

18167 

RECORD 

» 

29 

29 

8992 

RECORD 

# 

36 

39 

6441 

RECORD 

# 

39 

90 

7230 

RECORD 

# 

46 

63 

6987 

Figure  11.  Information  printed  out  concurrently  with  Envelope  Recognition 
(by  engagement).  The  record  numbers  listed  here  allow  access 
to  the  data  base  by  the  Data  listing  feature  of  the  PACE  pro¬ 
gram.  i\OT  is  presented  in  degrees;  Range  is  presented  in 
feet  (607*’  ft.  =  1  n.m.) 
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PILOT  SQURD  R>C  RIO 

Keycode*  F4  Keycode* 

RDV  SQ  R/C  MISSION  HINGMRN 

TR4  2V2  Keycode* 


DRTE 


T  START 

10:30:00 


TRPE  ENG 
585  1 

MODE 

2 


1 ST-RRDRR-R  1 ST-RRDBR-T  2ND-RRDRR-R  2ND-RRDRR-T  FORM 
1.21  18:36:28  .  :  : 


1  'ST  -  TRLLV-R  1  ST-TRLLY-T  1  ST-P  I  LOT-'R  1 0 
10892  10:37:04 

2ND-TRLLV-R  2ND-TRLLY-T  2ND-P I  LOT /R 1 0 

■  • 


PM  I  OFF  PM  I  NTRL  PMI  DEF 


FIGHTER  SHOTS: 


MISSLE  TIME 

ROT 

RRNGE 

IRS 

Vc 

RTfi 

RES 

9G  10:37:48 

100 

8024 

270 

367 

1 

N 

9G  10:39:46 

20 

5600 

410 

292 

2 

K 

RES 

N 


*  Note:  Operator  keycode  will  be  required  to  protect  identity  of 
aircrew  members. 

Figure  12.  Example  of  a  single  data  record  representing 
information  from  a  TACTS  ACM  engagement 
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PACE  Summary.  The  PACE  program  has  been  designed  to  provide  squadron 
level  performance  review  to  the  squadron  training  officer.  Engagements  are 
selected  by  squadron,  adversary  aircraft,  mission  type  and  dates.  Perfor¬ 
mance  review  occurs  for  several  critical  mission  phases,  including  Radar 
Contact,  Tally  Ho,  Tactics,  and  Envelope  Recognition.  Win/loss  statistics 
are  provided  by  an  additional  display,  Engagement  Outcomes.  An  automated 
debrief  capability,  or  feature,  aids  in  structuring  and  standardizing  the 
debrief  process.  Hardcopy  printout  is  available  for  these  review  phases. 

An  additional  feature,  Envelope  Recognition  (by  engagement),  enables  review 
of  additional  parameters  for  an  individual  shot,  and  the  Data  Listing  feature 
of  the  PACE  program  allows  access  to  any  single  engagement. 
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DATA  MANAGEMENT 

DATA  BASE.  The  data  base  is  a  file  of  sequential  records.  One  record  of 
information  is  added  to  the  data  base  for  each  engagement  a  pilot  flies  on 
TACTS.  A  list  of  the  information  stored  in  each  record  is  shown  in  Table  1, 
and  a  formatted  computer  printout  of  one  record  from  the  data  base  is  shown 
in  Figure  12.  Using  a  special  pointer  system  incorporated  into  PACE,  access 
to  the  data  base  is  allowed  on  the  four  data  fields  mentioned  earlier:  squad¬ 
ron  ,  mission  type ,  adversary  aircraft ,  and  date .  The  data  base  is  in  final 
form;  all  the  data  fields  needed  by  PACE  are  included  in  this  current  data 
base  system.  When  other  modules  are  added,  i.e.,  SOP,  TAD,  NORM,  pro¬ 
gramming  changes  to  the  data  base  or  to  the  data  management  program  may 
be  necessary. 

DATA  MANAGEMENT  PROGRAM.  "Entry"  is  the  name  of  the  data  management 
program.  Entry  is  net  considered  to  be  a  module  in  the  PAAS  program,  but 
is  a  separate  program  which  operates  independently  of  the  PAAS.  For 
security  purposes  Entry  is  to  be  used  only  by  personnel  assigned  the  task 
of  keeping  the  data  base  current  and  not  by  typical  users  of  PAAS  like  squad¬ 
ron  training  officers,  or  later  on,  pilots.  Entry  is  the  only  program  that  can 
be  used  to  change  the  data  base.  Its  most  important  function  is  to  add  new 
data,  although  it  can  also  be  used  to  look  at  records  of  specific  engagements, 
to  list  the  data  base,  and  to  change  data  that  have  been  added  incorrectly. 

Data  for  the  PAAS  data  base  is  transcribed  manually  onto  data  sheets 
and  then  entered  into  a  desktop  HP  45-T  computer  using  Entry.  Entry  is  an 
easy-to-use,  interactive  program  which  facilitates  efficient  data  entry.  Our 
long-range  plan  is  to  completely  automate  data  transfer  from  the  TACTS  com¬ 
puter  to  the  desktop  computer  now  located  at  Dunlap  and  Associates,  Inc., 
Western  Division.  However,  the  present  mode  of  data  transfer  is  highly 
efficient  and  minimally  time  consuming,  enabling  the  squadron  training  officer 
to  review  performance  on  the  same  day  an  engagement  takes  place. 
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TABLE  1.  TACTS  DEBRIEF  DATA  BASE  ELEMENTS 


ACM  MISSION  DATA 

• 

Pilot's  Name 

• 

Adversary  Squadron 

• 

Squadron 

• 

Adversary  Aircraft 

• 

Aircraft  Type 

• 

Mission  Type  (e.g.,  2v2) 

• 

RIO's  Name 

• 

Wingman's  Name 

• 

Date 

• 

Engagement  Start  Time 

• 

TACTS  Tape  No. 

• 

TACTS  Mode 

• 

Engagement  No. 

RADAR 

PROCEDURES 

• 

Contact  Range  -  1st  Bogey 

• 

Contact  Range  -  2nd  Bogey 

• 

Contact  Time  -  1st  Bogey 

• 

Contact  Time  -  2nd  Bogey 

•  Determine  Formation  (Y  or 


LOOKOUT  PROCEDURES 

•  Contact  Range  -  1st  Bogey  • 

•  Contact  Time  -  1st  Bogey  • 

MANEUVERING  INFORMATION 

•  Offensive  •  Neutral 

MISSILE  FIRE  INFORMATION 

•  Missile  Type  • 

•  Time  of  Shot  • 

•  AOT  • 

•  Range  • 

ADVERSARY  MISSILE  FIRE  INFORMATION 

•  Time  of  Shot  •  Result  (K  or  N) 


Contact  Range  -  2nd  Bogey 
Contact  Time  -  2nd  Bogey 

•  Defensive 

Shooter  IAS 
Closing  Velocity 
ATA 

Result  (K  or  N) 
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SUMMARY 

In  order  to  improve  the  training  effectiveness  of  the  Tactical  Aircrew 
Combat  Training  System  (TACTS) ,  a  computer  based  debrief  system  has  been 
proposed.  The  Performance  Assessment  and  Appraisal  System  (PAAS)  has 
been  designed  and  a  single  software  module.  Performance  Assessment  and 
Combat  Effectiveness  (PACE)  has  been  developed  thus  far.  PACE  is  a  stand¬ 
alone  and  independent  system  used  for  review  of  squadron  level  performance 
for  a  single  training  detachment.  Performance  data  collected  during  TACTS 
training  can  be  entered  in  PACE,  and  this  data  base  can  be  used  by  the 
PACE  programs  to  generate  statistical  summary  data  in  computer  graphic  dis¬ 
plays.  A  series  of  display  formats  for  selected  training  objectives  has  been 
designed  and  was  described  earlier. 

Future  development  of  PAAS  calls  for  three  additional  software  modules, 
Normative  Operational  Readiness  Measures  (NORM),  Training  Appraisal  and 
Diagnostics  (TAD),  and  Standard  Operating  Procedures  (SOP).  These  new 
modules  will  serve  respectively  to:  1)  review  Navy  group  or  fleet  normative 
data,  2)  provide  training  diagnostic  data,  and  3)  help  operational  training 
personnel  to  use  PAAS  in  the  most  effective  way.  The  availability  of  pro¬ 
grams  like  PACE,  or  better  yet,  the  total  integrated  PAAS  with  all  its  modules, 
will  serve  to  structure  air  combat  debriefs ,  while  providing  performance  based 
feedback  to  enhance  training. 

It  is  important  to  note  that  the  availability  of  performance  feedback  may 
help  aircrews  to  see  "out  of  tolerance'  missile  shots  and  to  this  extent  our 
debrief  has  been  directly  beneficial  to  envelope  recognition  training.  Other 
envelope  training  difficulties,  however,  may  require  additional  research  and/or 
supplementary  training  aids  as  outlined  in  the  next  report  section. 
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SECTION  III 

MISSILE  ENVELOPE  RECOGNITION 
PROBLEM  DEFINITION  SUMMARY 


INTRODUCTION 

BACKGROUND.  Ir.  spite  of  some  striking  advances  in  airborne  avionics,  and 
the  growing  sophistication  of  weapon  delivery  systems,  no  in-flight  fire  control 
system  has  evolved  to  provide  automatic  fire  solutions  for  current  Navy 
missiles.  The  determination  of  where  to  shoot,  in  terms  of  an  aircraft's  posi¬ 
tion  relative  to  a  target,  is  still  primarily  a  visual  tracking  task.  Selection 
of  correct  missile  delivery  boundaries,  referred  to  as  envelope  recognition,  is 
fundamentally  a  perceptual  task  in  which  a  pilot  must  estimate  the  position  of 
his  aircraft  with  respect  to  an  adversary  aircraft.  The  pilot  must  then  decide 
whether  his  relative  position  is  within  the  limits  of  a  prescribed  launch  cone, 
or  onvelope. 

The  Navy's  TACTS  was  developed  in  response  to  lessons  learned  in  Viet¬ 
nam  which  showed  that  U.S.  pilots  had  low  success  rates  in  air  combat  partly 
because  they  fired  their  weapons  outside  of  recommended  envelopes.  TACTS 
was  designed  to  improve  weapon  envelope  recognition  training,  as  well  as  to 
safely  train  and  evaluate  aircrews  in  all  aspects  of  ACM. 

Research  conducted  on  TACTS  over  the  past  five  years  ^  has  con¬ 
sistently  shown  that  pilots  are  highly  variable  in  their  ability  to  fire  missiles 
within  launch  envelopes  specified  by  tactical  doctrine.  More  recent  evidened® 
has  shown  that  missile  shots  are  most  frequently  fired  outside  of 


^See  footnotes  1,  2  and  3,  page  5 
16See  footnote  6,  page  7 
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recommended  boundaries.  Additionally,  this  finding  is  consistent  across 
weapon  types  and  aircrew  experience  levels. 

Performance  ata  collected  on  TACTS  have  demonstrated  that  the  fleet's 
most  experienced  aircrews,  many  with  extensive  combat  exposure  in  Vietnam, 
frequently  fire  out  of  thumbrule  training  envelope9--aven  when  told  that  the 
thumbrule  envelope  would  be  used  to  score  results  of  a  competitive  exercise. 
The  evidence,  provided  by  a  considerable  body  of  data  from  the  previously 
cited  research,  suggests  that  the  majority  of  aircrews  still  have  difficulty 
launching  weapons  within  prescribed  envelope  boundaries. 

The  Navy  is  now  introducing  new  all-aspect  weapons  to  the  fleet,  such 
as  the  radar  guided  AIM  7F  and  the  heat-seeking  AIM  9L.  If  these  particu¬ 
lar  missiles  meet  their  advertised  capabilities,  they  will  greatly  increase 
weapon  effectiveness  in  the  fleet  by  expanding  acceptable  launch  zones  from 
a  target's  rear  hemisphere  to  an  all-aspect  application. 

Actual  use  of  the  all-aspect  weapons,  howe/er,  will  still  require  pilots 
to  fly  to,  recognize,  and  fire  within  prescribed  weapon  envelopes  which 
maximize  kill  probability.  Furthermore,  pilots  will  now  have  a  wider  variety 
of  weapons  in  their  inventory  and  onboard  their  aircraft  from  which  to 
choose.  The  net  result  is  that  pilots  will  not  only  have  to  determine  where 
to  shoot  but  also  which  weapon  to  shoot  during  the  course  of  an  air  combat 
engagement.  Knowledge  of  various  missile  envelopes  (9G/H,  9L,  7F) ,  and 
skill  in  firing  these  weapons,  will  continue  to  be  essential  to  a  pilot's 
effective  use  of  available  weapons. 

The  recent  advances  in  weapon  systems  reflect  a  prevailing  philosophy 
that  the  application  of  high  technology  may  eliminate  or  substantially  dimin¬ 
ish  problems  associated  with  fleet  training.  Evidence  providod  in  our 
problem  definition  study  ^  ^suggests  that  the  mere  application  of  technological 


^See  footnote  6,  page  7 
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innovation,  exemplified  by  TACTS  and  new  weapon  advances,  has  not  been 
sufficient  to  eliminate  the  need  for  improved  fleet  training  practices.  In 
spite  of  a  six-year  availability  of  TACTS,  and  improvements  in  weapon  sys¬ 
tem  capabilities,  fleet  performance  in  envelope  recognition  remains  below 
standards  set  by  Navy  doctrinal  recommendations.  The  effective  use  of  all 
weapons  in  the  fleet  inventory  ultimately  rests  with  the  pilot,  given  his 
understanding  of  weapon  capabilities  and  their  proper  application.  The 
requirement  for  pilots  to  learn  several  missile  envelopes  and  to  gain  experi¬ 
ence  firing  new  weapon  types,  in  actuality,  may  have  further  complicated 
the  training  process. 

PURPOSE  AND  SCOPE.  In  view  of  the  potential  difficulties  associated  with 
missile  envelop-',  recognition  training,  the  Naval  Training  Equipment  Center, 
Orlando,  Florida  sponsored  a  problem  definition  study  in  order  to  identify 
any  training  deficiencies  which  could  be  corrected  through  improved  instruc¬ 
tional  methods. 

This  problem  definition  study  is  now  complete  and  recommendations  for 
improving  envelope  recognition  training  have  been  proposed J 8  The  summary, 
presented  here,  delineates  some  of  the  more  salient  envelope  recognition 
training  deficiencies  identified  in  the  problem  definition  study  and  reviews 
our  research  conclusions  and  recommendations.  It  should  be  noted  that 
much  technical  detail  of  our  findings  must  be  left  out  of  this  unclassified 
summary  of  the  original  study.  The  reader  interested  in  these  details  is 
urged  to  consult  the  more  comprehensive  problem  definition  study  previously 
referenced. 


18See  footnote  6,  page  7 
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PROBLEM  DEFINITION  SUMMARY 

Major  problems  associated  with  U.S.  Navy  missile  envelope  recognition 
training  fail  into  four  broad  categories  os  follows: 

1.  Missile  envelope  derivation  and  application , 

2.  TACTS  utilization  and  training  feedback, 

3.  Envelope  concepts  and  instructional  methods,  and 

4.  Training  impact  of  new  weapons. 

MISSILE  ENVELOPES.  Missile  envelopes  are  derived  from  mathematical 
models  which  simulate  missile  launch  and  guidance  characteristics.  On  the 
west  coast  TACTS,  missile  launch  results  (kill,  no-kill)  are  based  on  kine¬ 
matic  mathematical  models  which  do  not  consider  such  factors  as  warhead 
lethality  and  target  vulnerability.  Thumbrule  boundaries,  which  represent 
the  heart-of-the-envelope  (i.e.,  most  effective  launch  zones)  are  also 
derived  from  kinematic  mathematical  models. 

There  is  considerable  controversy  in  the  operational  community  con¬ 
cerning  the  credibility  of  the  kinematic  simulations,  and  the  incorporation 
oi  ,  eir  resulting  launch  boundary  data  in  the  TACTS.  Scoring  results 
from  TACTS  are  notoriously  liberal  in  uwarding  kills  and  this  issue  raises 
questions  regarding  the  true  capabilities  of  fleet  weapons,  and  their 
appropriate  application . 

To  further  complicate  the  issue,  the  use  of  various  thumbrule  training 
envelopes  vis  a  vis  TACTS  scoring  methods  results  in  highly  disparate 
missile  fire  success  rates.  Aircrews  vary  widely  in  their  viewpoints  con¬ 
cerning  the  thumbrule  limits  and  their  use  during  TACTS  training.  Some 
aircrews  prefer  to  capitalize  on  the  full  missile  envelope,  represented  by 
TACTS  Mode  5,*  while  others  train  to  various  thumbrule  standards  to 
emphasize  heart-of-envelope  boundaries. 

♦TACTS  can  operate  in  five  different  modes.  Modes  1-3  provide  thumbrule 
missile  boundaries  and  are  designed  primarily  for  envelope  recognition  train¬ 
ing.  Modes  4  and  5  provide  dynamic  missile  launch  simulation  with  full 
envelope  boundaries.  (See  references  for  more  detail.) 
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TACTS  UTILIZATION.  Extensive  experience  (over  the  past  five  years)  on 
TACTS  indicates  that  pilots  seldom  use  early  TACTS  modes  (1-3)  which  are 
designed  to  teach  missile  envelope  boundaries.  Most  aircrews  prefer  to 
train  in  Mode  5  because  they  believe  that  this  training  configuration  is  more 
realistic  for  full  spectrum  air  combat  training,  i.e..  Mode  5  is  more  realistic 
in  terms  of  missile  capability  and  its  allowance  for  target  evasive  maneuvers. 
As  a  result,  aircrews  do  not  focus  their  training  specifically  on  teaching 
missile  launch  zones.  Digital  information  (e.g.,  range,  angle-off -tail ,  etc.) 
is  available  to  aircrews  during  the  TACTS  replay,  but  the  digital  data  format 
may  not  be  the  most  suitable  way  to  represent  missile  envelopes.  Graphic 
methods  that  depict  an  envelope  trailing  a  target  may  be  more  appropriate 
for  debrief  training  feedback,  and  would  visually  illustrate  shots  in  and  out 
of  envelope. 

TACTS  training,  as  currently  practiced,  does  not  provide  aircrews  with 
many  repeated  missile  fire  opportunities.  Based  on  data  taken  over  300 
engagements,  fighters  fire  an  average  of  three  shots  per  engagement. 

Usually,  the  three  or  so  shots  are  distributed  across  several  participating 
aircraft.  Exposure  to  repeated  missile  fire  opportunities,  and  appropriately 
formatted  knowledge  of  results,  are  considered  by  operational  aircrews  to  be 
essential  requirements  for  acquiring  envelope  recognition  skills.  TACTS 
resource  restrictions,  i.e.,  fuel  considerations  and  TACTS  time  limits,  may 
preclude  the  use  of  the  range  specifically  to  train  envelope  recognition. 
Aircrews  are  probably  correct  in  their  decision  to  maximize  the  time  on  range 
for  full  spectrum  ACM. 

INSTRUCTIONAL  METHODS.  There  are  considerable  difficulties  related  to 
the  conceptualization  of  missile  envelopes  and  the  methods  used  to  instruct 
pilots  in  proper  launch  boundaries.  Missile  launch  boundaries,  in  reality, 
vary  widely  over  the  dynamic  range  of  air  combat  engagement  conditions. 

The  actual  envelope  may  expand,  contract,  warp  and  distort  at  various 
engagement  speeds  and  target  maneuvering  situations.  Simplified  repre¬ 
sentations,  such  as  thumbrule  envelopes,  are  designed  to  meet  the  most 


34 


NAVTRAEQUIPCEN  78-C-0136-3 


frequently  encountered  engagement  conditions,  but  at  best  are  static, 
imprecise  estimates  of  the  true  weapon  capabilities.  TACTS  provides  sim¬ 
ulations  based  on  the  dynamic  aspects  of  an  engagement  (i.e.,  target- shooter 
relationship)  but  the  information  related  to  the  pilot  as  a  simulation  output 
is  a  simplified  kill /no-kill  indication.  A  pilot  does  not  see  a  representa¬ 
tion  of  how  a  missile  envelope  changes  under  dynamic  conditions  of  the 
engagement.  In  brief,  a  pilot  must  estimate  launch  boundaries  based  on 
static  thumbrule  restrictions,  which  depict  an  imaginary  launch  cone,  or 
TACTS,  which  gives  no  visual  representation  of  the  envelope  whatsoever. 

Furthermore,  the  idea  of  a  launch  cone  is  not  readily  transferred  to 
the  actual  judgement  of  launch  conditions  in  the  air,  during  which  a  pilot 
estimates  his  aircraft  position  based  on  a  maneuvering  target  (i.e.,  by 
seeing  target  element  details  such  as  apparent  size  and  wing  and  fuselage 
exposed  to  view). 

TRAINING  IMPACT  OF  NEW  WEAPONS.  New  weapon  development,  such  as 
the  radai  guided  AIM  7F  and  the  heat- seeking  AIM  9L  were  designed  to 
simplify  ti  e  weapon  launch  process  by  providing  expanded  zones  of  missile 
effectiveness.  On  the  surface,  these  technological  innovations  appear  to 
eliminate  problems  related  to  envelope  recognition  training.  In  the  words 
of  some  of  the  operational  aircrews,  these  new  weapons  have  been  dubbed 
magic  missiles.  But  are  they? 

The  highly  touted  advances  of  the  AIM  7F  and  AIM  9L  are  as  yet  only 
advertised  capabilities  which  have  not  stood  the  test  of  operational  applica¬ 
tion.  The  history  of  advertised  weapon  capabilities  was  shown  to  be  wrong 
in  the  past,  with  the  notorious  failure  of  tht  old  version  Sparrow  (AIM  7E) 
as  a  case  in  point.  A  question  of  highest  priority  in  the  minds  of  the 
authors  and  operational  aircrews  is,  can  we  afford  to  be  wrong  in  the  future 
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Evidence  presented  in  the  problem  definition  study  suggests  that  even 
with  the  application  of  high  technology,  which  includes  a  six-year  availability 
of  TACTS  and  improvements  in  weapon  capabilities,  fleet  performance  remains 
below  standards  set  by  Navy  doctrinal  recommendations. 

The  effective  use  of  an  weapons  in  the  fleet  inventory  ultimately  rests 
with  the  pilot ,  given  his  understanding  of  weapon  capability  and  their 
proper  application.  A  requirement  for  pilots  to  learn  several  missile  enve¬ 
lopes  and  to  gain  experience  firing  new  weapon  types  has  complicated  the 
training  process. 

Finally,  attaining  the  full  potential  in  applying  state-of-the-art  missile 
improvements  (AIM  7F ,  AIM  9L)  may  not  be  possible  because  of  an  opera¬ 
tional  requirement  to  positively  identify  unfriendly  aircraft  (i.e.,  pilots  may 
not  be  able  to  identify  enemy  aircraft  at  maximum  ranges) .  By  the  time 
positive  visual  identification  of  an  adversary  is  made,  the  friendly  aircraft 
is  approaching  minimum  range  limits  of  the  weapon  system. 

CONCLUSIONS  AND  RECOMMENDATIONS 

All  of  the  problem  areas  discussed  in  the  previous  section  are  con¬ 
sidered  to  be  contributing  factors  to  the  envelope  recognition  performance 
variation  observed  in  the  fleet  today.  Unless  steps  are  taken  to  rectify 
some  or  all  of  these  difficulties,  envelope  recognition  training  effectiveness 
(and  perhaps  future  operational  weapon  employment)  may  be  seriously  com¬ 
promised. 

The  following  training  improvements  are  recommended: 

1.  Define  more  realistic  and  credible  missile  envelope 
thumbrules. 

2.  Provide  structured  debrief  data  which  visually  illustrate 
launch  envelopes  to  aircrews. 

1  9 

See  footnote  6,  page  7 
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3.  Develop  training  aids  to  teach  dynamic  missile  envelope 
boundaries  to  aircrews,  instead  of  static  thumbrule 
representations. 

4.  Study  the  use  of  existing  ACM  simulators,  or  develop 
a  special  purpose  part  ta3k  envelope  trainer,  so  that 
pilots  can  practice  envelope  recognition  over  repeated 
trials  (including  missile  fire  with  knowledge  of  results 
presented  in  a  visually  enriched  format). 

5.  Evaluate  the  use  of  earlier  TACTS  training  modes  (1-3) 
with  the  inclusion  of  more  realistic  rule-of-thumb 
envelopes . 

Each  of  these  recommendations  should  be  viewed  in  the  light  of  their 
likely  cost  and  technical  feasibility,  in  order  to  arrive  at  the  most  cost 
effective  solution(s)  to  the  envelope  recognition  training  problem. 


37 


NAVTRAEQUIPCEN  78-C-0136-3 


REFERENCES 


Brictson,  C.A.,  Ciavareili,  A.P.  and  Jones,  T.N.  Development  of  Air¬ 
crew  Performance  Measures  for  the  Air  Combat  Maneuvering  Range  ~ 
(A<3MR)  (0).  Pensacola,  Florida:  Naval  Aerospace  Me  diced  Research 
Laboratory,  Report  No.  L53001,  June  1977.  (CONFIDENTIAL) 

Brictson,  C.A.,  Ciavareili,  A.P.,  Pettigrew,  K.W.  and  Young,  P.A. 
Performance  Assessment  Methods  and  Criteria  for  the  Air  Combat 
Maneuvering  Range  (ACMft):  Missile  Envelope  Recognition  (U). 
Pensacola,  Florida:  Naval  Aerospace  Medical  Research  Laboratory , 
Special  Report  No.  7E-4,  July  1978.  (CONFIDENTIAL) 

Ciavareili,  A.P.,  Brictson,  C.A.,  and  Young,  P.A.  Development  and 
Application  of  Performance  Criteria  md  Aircrew  Assessment  Methods 
for  the  Air  Combat  Maneuvering  Range  (ACMR)  (U).  Pensacola, 
Florida:  Naval  Aerospace  Medical  Research  Laboratory,  Special  Report 
79-5,  September  1979.  (CONFIDENTIAL) 

Ciavareili,  A.P.,  Pettigrew,  K.W.  and  Brictson,  C.A.  Development 
of  a  Computer  Based  Air  Combat  Maneuvering  Range  Debrief  System: 
Interim  Report  (Volume  ij^  La  Jolla,  California:  Dunlap  and 
Associates,  Inc.,  January  1980. 

Ciavareili,  A.P.,  Narsete,  E.M.,  and  Brictson,  C.A.  A  Problem  Defi¬ 
nition  Study  of  U.S.  Navy  Air-to-Air  Missile  Envelope  Recognition 
Training  (U):  Interim  Report  (Volume  II):  La  Jolla ,  California : 
Sunlap  and  Associates,  Inc.,  April  1980.  (CONFIDENTIAL) 

Ciavareili,  A.P.  Development  and  application  of  aircrew  performance 
criteria  and  assessment  methods  for  TACTS.  Proceedings  of  the 
Seventh  TACTS /ACMI  Users  Seminar.  San  Diego,  California: 

NAV \IRSYSCOM,  May  20-21,  1980. 

Williams,  A.M,  Performance  assessment  and  combat  effectiveness 
(PACE).  Proceedings  of  the  Seventh  TACTS /ACMI  Users  Seminar. 

San  Diego,  California :  NAVAIRSYSCOM ,  May  20-21,  1980. 


NAVTRAEQUIPCEN  78-C-01 36-3 


DISTRIBUTION  LIST 


Commanding  Officer 
Naval  Training  Equipment  Center 
Orlando,  FL  32813  61 

Defense  Technical  Information  Center 
Cameron  Station 

Alexandria,  VA  22314  12 

All  other  addressees  receive  one  copy 

Commanding  Officer 
U.S.  Army  Security  Agency 
Training  Center  Library 
Ft.  Devens,  MA  01433 

Headquarters  ESD/DRI 
Hanscom  AFB,  MA  01731 

Commanding  Officer 
Navy  Submarine  Base  New  London 
Attn:  Psychology  Section,  Box  600 
Groton,  CT  06340 

National  Aviation  Facilities 
Experimental  Center 
Libra ry 

Atlantic  City,  NJ  08405 

Dr.  Donald  W.  Connolly 
Research  Psychologist 
Federal  Aviation  Administration 
FAA  NAFEC  ANA-230  Bldg  3 
Atlantic  City,  NJ  08405 

Superintendent 

U.S.  Military  Academy  Library 
West  Point,  NY  10996 

Commanding  Officer 

Rome  Air  Development  Center 

Library  ( TSLD ) 

Griffiss  AFB,  NY  13440 

Commander 

Naval  Air  Development  Center 
Attn:  Code  6022 
Warminster,  PA  18974 

LCDR  Steve  Harris 

Naval  Air  Development  Center 

Code  6021 

Warminster,  PA  18974 

1  of 


Chief 

U.S.  Army  Strategy  and  Tactics  Group 
8120  Woodmont  Ave. 

Bethesda,  MD  20014 

Commanding  Officer 

Air  Force  Office  of  Scientific  Research 
Technical  Library 
Washington,  D.C.  20025 

Technical  Library 
DDR&E 

Room  3C122,  The  Pentagon 
Washington,  D.C.  20301 

Director 

Defense  Research  and  Engineering 
Washington,  D.C.  20301 

OUSDR&E  (R&AT)  (E&LS) 

CDR  Paul  R.  Chatelier 
Washington,  D.C.  20301 

Chief,  Research  Office 
Office  Deputy  Chief  of  S+'ff  for 
Personnel 

Department  of  Army 
Washington,  D.C.  20310 

U.S.  Army  Element 

Inter  American  Defense  College  Library 
Ft.  Leslie  J.  McNair 
Washington,  D.C.  20315 

HQAFSC/DLS 
Andrews  AFB 

Washington,  D.C.  20334 

Chief  of  Naval  Operations 
OP-115 

Research,  Development  and  Studies 
Room  G836 

Washington,  D.C.  20350 

Chief  of  Naval  Operations 
0P-112C1 

Washington,  D.C.  20350 

Assistant  Secretary  of  the  Navy 
Research,  Engineering  and  Systems 
Washington,  D.C.  20350 

6 


infill  rn  fc*  i«l  Vi inv.i<  ■  V  l 


NAVTRAEQUI PCEN  78-C-01 36-3 


Office  of  Deputy  Chief  of  Naval  Commander 

Operations  Naval  Sea  Systems  Command 

Manpower,  Personnel  &  Training  (OP-Ol)  Technical  Library 


Washington,  D.C.  20350 

Chief  of  Naval  Operations 
Attn:  Dr.  R.  G.  Smith/0P-987H 
Washington,  D.C.  20350 

Chief  of  Naval  Operations 
0P-596C 

Washington,  D.C.  20350 

Chief  of  Naval  Operations 
Washington,  D.C.  20350 

Chief  of  Naval  Operations 
0P-593B 

Washington,  D.C.  20350 

Chief  of  Naval  Material 
MAT  031 M 

Washington,  D.C.  20360 

Chief  of  Naval  Material 
MAT  08D2 
CP5,  Room  678 

Attn:  Arnold  I.  Rubinstein 
Washington,  D.C.  20360 

Commander 
Naval  Electronic  Systems  Command 
Code  03 

Washington,  D.C.  20360 
Comma rder 

Naval  Air  Systems  Command 
Technical  Library 
AIR-9500 

Washington,  D.C.  20361 
Commander 

Naval  Ai^  Systems  Command 
Washington,  D.C.  20361 

Commander 

Naval  Air  Systems  Command 
AIR  34 OF 

Washington,  D.C.  20361 
Commander 

Naval  Air  Svstems  Command 
AIR  413F 

Washington,  D.C.  20351 


SEA  99612 

Washington,  D.C.  20362 
Commander 

Naval  Sea  Systems  Command 
Code  61R2/Mr.  P.  J.  Andrews 
Washington,  D.£.  20362 

Commander 

Naval  Sea  Systems  Command 
Code  315/Chief  Sci  Randd 
Washington,  D.C.  20362 

Commander 

Naval  Sea  Systems  Command 
Code  335 

Washington,  D.C.  20362 

Bureau  of  Naval  Personnel 
Assistant  Chief  for  Education 
and  Training 
Personnel  CM 
Washington,  D.C.  20370 

Director,  Personnel  and  Training 
Analysis  Office 
Building  200-3 
Washington  Navy  Yard 
Washington,  D.C.  20374  • 

Naval  Researcn  Laboratory 
Attn:  Library 
Washington,  D.C.  20275 

Commander 

Naval  Supply  Systems  Command 
Code  03 

Washington,  D.C.  20376 

Hq  Marine  Corps 

Code  APC/LTC  J.  W.  Biermas 

Washington,  D.C.  20380 

Scientific  Adviser 
Headquarters  U.S.  Marine  Corps 
Washington,  D.C.  203t>0 

Commandant  of  the  Marine  Corps 
Code  0TTF  31 
Washingtor,  D.C.  20380 


2  of  6 


NAVTRAEQUI PCEN  78-C-01 36-3 


Scientific  Technical  Information 
Office 
NASA 

Washington,  D.C.  20546 

Federal  Aviation  Administration 
Technical  Library 
Bureau  Research  and  Development 
Washington,  D.C.  20590 

Commander 

Naval  Air  Test  Center 
CT  176 

Patuxent  River,  MD  20670 

Dr.  Sam  Schiflett 
Naval  Air  Test  Center 
SY  721 

Patuxent  River,  MD  20670 

Director  Educational  Development 
Academic  Computing  Center 
U.S.  Naval  Academy 
Annapol is,  MD  21402 

Commanding  Officer 
U.S.  Army  Engineer  Research  and 
Development  Laboratories 
Attn:  Library 
Ft.  Bel  voir,  VA  22060 

Dr.  Jesse  Orlansky 
Institute  for  Defense  Analyses 
Science  and  Technology  Division 
400  Army-Navy  Drive 
Arlington,  VA  22202 

Defense  Adv.  Research  Projects  Agency 
Information  Processing  Techniques  Ofc 
1 400  Wi 1  son  Bl vd . 

Arlington,  VA  22209 

Defense  Adv.  Research  Projects  Agency 
Cybernetics  Technology  Office 
1400  Wilson  Boulevard 
Arlington,  VA  22209 

Chief  of  Naval  Research 
Director,  Air  Programs 
Code  210 

Arl ington,  VA  2221  7 


Dr.  Marshall  J.  Farr 
Director,  Persornel  and  Training 
Research  Program  (Code  458) 

Office  of  Naval  Research 
Arlington,  VA  22217 

Chief  of  Naval  Research 
Code  455 

800  N.  Quincy  St. 

Arlington,  VA  22217 

Chief  of  Naval  Research 
Code  458 

800  N.  Quincy  St. 

Arlington,  VA  22217 

Mr.  Jerry  Malecki 
Office  of  Naval  Research 
Code  455 

800  N.  Quincy  St. 

Arl ington,  VA  22217 

Dr.  Henry  M.  Halff 
Office  of  Naval  Research 
Code  458 

Arl ington ,  VA  22217 
Commanding  Officer 

Fleet  Combat  Training  Center  Atlantic 
Attn:  Mr,  Hartz,  Code  02A 
Darn  Neck 

Virginia  Beach,  VA  23461 

Commander 
Naval  Air  Force 

U.S.  Atlantic  Fleet  (Code  312  E) 

NAS  Norfolk 
Norfol k ,  VA  2351  1 

Commanding  Officer 
FASOTRAGRULANT  (50) 

NAS  Norfolk  VA  23511 

Army  Training  Support  Center 
ATTSC-DS 

Ft.  Eustis,  VA  23604 

Conrad  Technical  Library 

U.S.  Army  Signal  Center  at  Ft.  Gordon 

Building  29807 

Ft.  Gordon,  GA  30905 


3  of  6 


NAVTRAEQUI PCEN  78-C-01 36-3 


Chief,  ARI  Field  Unit 
P.O.  Box  2086 
Attn:  Librarian 
Ft.  Benning,  6A  31905 


USAHEL/USAAVNC 

Attn:  DRXHE-FR  (Dr.  Hofmann) 

P.O.  Box  476 

Ft.  Rucker,  AL  36362 


LSO  Training  Model  Manager 
Officer  in  Charge 
LSO  School 
Box  171 

NAS  Cecil  Field,  FL  32215 

Vision  Research  Division 
Code  32 

Naval  Aeromedical  Research  Lab 
Naval  Air  Station 
Pensacola,  FL  32508 

Chief  of  Naval  Education 
Code  017 

NAS  Pensacola,  FL  32508 

Chief  of  Naval  Education 
Code  N-2/CAPT  Bauchspies 
Pensacola,  FL  32508 

Commanding  Officer 
Naval  Education  Training  Program 
and  Development  Center 
Attn:  Technical  Library 
Pensacola,  FL  32509 

Chief  of  Naval  Education  and  Training 
Code  010 

Pensacola,  FL  32509 

Selection  and  Training  Division 
Department  of  Psychology 
Code  26 

Naval  Aerospace  Medical  Research  Lab 
Pensacola,  FL  32512 

TAWC/TN 

Eglin  AFB ,  FL  32542 
Di  rector 

Air  University  Library 
Maxwell  AFB,  AL  36100 

Commanding  Officer,  U.S.  Army 
Aviation 

Air  Traffic  Control  School 
ATZQ-T-AT-ATC 
P.O.  Box  385 
Ft.  Rucker,  AL  36362 


Chief,  ARI  Field  Unit 

P.O.  Box  476 

Ft.  Rucker,  AL  36362 

Commanding  Officer 

Naval  Air  Technical  Training  Center 

Code  104,  Building  S-54 

NAS  Memphis  (85) 

Millington,  TN  38054 


CW02  Ray  Priest  (Code  74111 
Naval  Air  Technical  Training  Center 
NAS  Memphis  (85) 

Millington,  TN  38054 

Mr.  Harold  A.  Kottmann 
ASD/YWE 

Wright  Patterson  AFB,  OH  45433 

Air  Force  Institute  Technology  Library 
Wright-Patterson  AFB,  OH  45433 

ASD/ENESS 

Attn:  R.  B.  Kuhnen 
Wright-Patterson  AFB,  OH  45433 

Air  Force  Human  Resources  Laboratory 
AFHRL/LR 

Logistics  Research  Division 
Wright-Patterson  AFB,  OH  45433 

6570  AMRL/HE 

Wright-Patterson  AFB,  OH  45433 

ASD/ENETC 

Mr.  R.  G.  Cameron 

Wright-Patterson  AFB,  OH  45433 

Commanding  Officer 

Naval  Hospital  Corps  School 

Great  Lakes ,  IL  60088 
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Chief  of  Naval  Technical  Training 
Code  0161 
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Commandant 

U.S.  Army  Command  and  General  Staff 
College 

Library  Division 

Ft.  Leavenworth,  KS  66G27 

Chief  of  Naval  Reserve 

Code  S- 331 1 

New  Orleans,  LA  70146 

Headquarters  34  Tactical  Airlift 
Training  Group/TTDI 
Little  Rock  AFB ,  AK  72076 

Federal  Aviation  Administration 
AAC-954C 

Aeronautical  Center,  Flight  Standards 
Branch 

Oklahoma  City,  OK  73101 

Chief,  Methodology  and  Standards 
Staff 

Federal  Aviation  Administration 
Academy 

Aeronautical  Center,  AAC  -  914 
P.0.  Box  25082 
Oklahoma  City,  OK  73125 

Commandant 

U.S.  Army  Field  Artillery  School 
Counterfire  Department 
Attn:  Eugene  C.  Rogers 
Ft.  Sill  ,  OK  73503 

Commandant 

U.S.  Army  Field  Artillery  School 
ATSF-TD-TS  (Mr.  Inman) 

Ft.  Sill  ,  OK  73503 

Headquarters 

Air  Training  Command,  XPTI 
Attn:  Mr.  Goldman 
Randolph  AFB,  TX  78148 

Commanding  Officer 
School  of  Aviation  Medicine 
Aeromed  Library 
Brooks  AFB 

San  Antonio,  TX  78200 
AFHRL/MF 

Brooks  AFB,  TX  78235 


Stimson  Library 

Academy  of  Health  Sciences 

U.S.  A; my 

Attn-  Mss  H.  T.  Morrow,  DOCS  Librarian 
Ft.  Sai  Houston,  TX  78234 

U.S.  Air  corce  Human  Resources  Lab 
AFHRL-MPh 

Manpower  and  Personnel  Division 
Manpower  ind  Force  Management  Systems 
Branch 

Brooks  AFB,  TX  78235 

U.S.  Air  force  Human  Resources  Lab 
TSZ 

Brooks  AFB..  TX  78235 
ATJlrtL/PE 

Brooks  AFB,  TX  78235 

Chief  of  Naval  Air  Training 
Attn:  Code  3146  (LSO) 

'AS  Corpus  Christi,  TX  78419 

Chief  of  Naval  Air  Training 
ATTN:  Code  333 

NAS  Corpus  Christi,  TX  78419 
Superintendent 

U.S.  Air  Force  Academy  Library 
Code  DFSLB-D 
Denver,  CO  80840 

U.S.  Air  Force  Human  Resources  Lab 
AFHRL-OT  (Dr.  Rockway) 

Williams  AFB,  AZ  85224 

Commanding  Officer 
Human  Resources  Laboratory 
Operational  Training  Division 
Williams  AFB,  AZ  85224 

U.S.  Air  Force  Human  Resources  Lab 
AFHRL-FT  (Dr.  Edwards) 

Flying  Training  Division 
Williams  AFB,  AZ  85224 

Chief  of  Naval  Education  and  Training 
Liaison  Office 
Human  Resources  Laboratory 
Flying  Training  Division 
Williams  AFB,  AZ  85224 
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U.S.  Air  Force  Human  Resources  Lab 
AFHRL-OT 

Operational  Training  Division 
Williams  AFB ,  AZ  85224 

AFHRL/OTO 

Luke  AFB,  AZ  85309 
Commanding  Officer 

Naval  Education  and  Training  Support 
Center,  Pacific 
Code  N5B  (Mr.  Rothenberg) 

San  Diego,  CA  92132 

Commander,  Naval  Air  Force 
U.S.  Pacific  Fleet  (Code  311  L) 

NAS  North  Island 
San  Diego,  CA  92135 

Commander,  Naval  Air  Force 
U.S.  Pacific  Fleet  (Code  342) 

NAS  North  Island 
San  Diego ,  CA  921 35 

Commander,  Naval  Air  Force 
U.S.  Pacific  Fleet  (Code  316) 

NAS  North  Island 
San  Diego,  CA  92135 

Commanding  Officer 
Fleet  Training  Center 
Attn:  Training  Department 
Naval  Station 
San  Diego,  CA  92136 

Commander,  Training  Command 
Attn:  Educational  Advisor 
U.S.  Pacific  Fleet 
San  Di ego ,  CA  92147 

Commanding  Officer 

Fleet  Combat  Training  Center,  Pacific 
Code  09A 

San  Diego,  CA  92147 
Commanding  Officer 

Fleet  Anti-Submarine  Warfare  Training 
Center,  Pacific 
Attn  :  Code  001 
San  Diego,  CA  92147 

Commanding  Officer 

Naval  Health  Research  Center 

San  Diego,  CA  92152 
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Commander 

Naval  Electronics  Lab  Center 

Attn:  Library 

San  Diego,  CA  92152 

Navy  Personnel  Research  and  Development 
Center 

Attn:  M.  McDowell 
Library,  Code  P201L 
San  Diego ,  CA  921  52 

Commander 

Pacific  Missile  Test  Center 
Point  Mugu,  CA  93042 

LT  Wayne  R.  Helm 

Human  Factors,  Engineering 

Code  1226 

Point  Mugu,  CA  93042 

National  Aeronautical  and  Space 
Administration 

High  Speed  Research  Center  Library 
Edwards  AFB,  CA  93523 

Commanding  Officer 

Air  Force  Flight  Test  Center 

FTOTL 

Technical  Library  Branch 
Edwards  AFB,  CA  93523 

Commander 

Naval  Weapons  Center 
Code  3154  (Mr.  Bob  Curtis) 

China  Lake,  CA  93555 

Plans  Officer  Psychologist 
Ft.  Ord,  CA  93941 

National  Aeronautical  and  Space 
Administration 

Ames  Research  Center  Aircraft 
Inspection  Branch 
Mail  Stop  211-5 
Moffett  Field,  CA  95050 


